A Survey of Imagery Techniques for Semantic Labeling of Human- Vehicle Interactions in Persistent Surveillance Systems

Semantic labeling of Human-Vehicle Interactions (HVI) helps in fusion, characterization, and understanding cohesive patterns that when analyzed and reasoned, they may jointly reveal pertinent threats. Various Persistent Surveillance System (PSS) imagery techniques have been proposed in the past for identifying human interactions with various objects in the environment. Understanding of such interactions facilitates to discover human intentions and motives. However, without consideration of circumstantial context, reasoning and analysis of such behavioral activities is a very challenging and difficult task. This paper presents a current survey of related publications in the area of context-based HVI, in particular, it discusses taxonomy and ontology of HVI and presents a summary of reported robust imagery processing techniques for spatiotemporal characterization and tracking of human target in urban environments. The discussed techniques include model-based, shape-based and appearance-based techniques employed for identification and classification of objects. A detailed overview of major past research activities related to HVI in PSS with exploitation of spatiotemporal reasoning techniques applied to semantic labeling of the HVI is also presented.